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IT  abstract  '\m'jrn  :00  i'.vr::'' 

This  document  describes  the  Testbank  program,  which  is  designed  to  enhance  the 
reliability  and  efficiency  of  test  development.  Testbank  accomplishes  this  goal  by 
maintaining  a  data  base  of  test  items  together  with  reference  data  for  each  time  the 

item  has  been  used.  A  test  developer  can  then  query  the  test  database  for  items 

that  satisfy  any  number  of  query  parameters.  A  full  range  of  test  editing  functions 

are  available  to  modify  and  revise  the  test.  Upon  completion  of  a  test,  the 

Testbank  software  will  create  a  ChiWriter  document  of  the  test.  In  order  to  enhance 
the  reliability  of  the  produced  test,  the  Testbank  software  will  use  the  reference 
data  from  the  included  items  to  predict  the  mean  and  standard  deviation  for  the 
produced  test. 
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1.0  INTRODUCTION 


1.1  Purpoae.  The  Teatbank  program  is  designed  to  enhance  the  reliability  and 
efficiency  of  teat  development.  Teatbank  accomplishes  this  goal  by 
maintaining  a  data  base  of  teat  items  together  with  reference  data  for  each 
time  the  item  has  been  used.  A  teat  developer  can  than  query  the  test  data 
base  for  items  that  satisfy  any  number  of  query  parameters.  A  full  range  of 
test  editing  functions  are  available  to  modify  and  revise  the  test.  Upon 
completion  of  a  test,  the  Teatbank  software  will  create  a  ChiWriter  document 
of  the  test.  In  order  to  enhance  the  reliability  of  the  produced  test,  the 
Teatbank  software  will  use  the  reference  data  from  the  included  items  to 
predict  the  mean  and  standard  deviation  for  the  produced  test. 

In  support  of  the  test  development  functions,  Teatbank  provides  a  full  range 
of  item  maintenance  functions.  These  functions  allow  items  to  be  entered  from 
either  existing  files,  directly  from  ChiWriter,  or  by  modifying  existing 
items.  Existing  items  may  also  be  improved  with  their  evolution  recorded. 
The  reference  data  for  items  may  be  modified  and  updated.  A  number  of  printed 
reports  are  also  available  for  reference. 

1.2  Audience.  This  document  is  intended  for  several  audiences  to  include: 

-  Test  Developers:  Those  building  new  teats  with  items  from  the  test  data 
bases . 


-  Test  Item  Maintainers:  Those  putting  items  and  reference  data  into  the 
test  data  bases. 

-  Teatbank  Administrators:  Those  overseeing  the  integrity  of  the  test  data 
bases . 

1.3  Hardware  Requirements.  Testbank  requires  an  IBM  PC/XT/AT-compatible  or 
386  processor  with  640K  bytes  of  memory.  (If  integrated  use  of  ChiWriter  is 
not  required,  only  384K  bytes  are  needed.)  If  additional  Expanded  Memory  is 
available,  it  will  be  used  to  cache  overlays  and  speed  up  the  program. 
Storage  of  the  teat  items  requires  a  hard  disk  with  approximately  IM  byte  of 
storage  for  every  100  test  items.  A  color  monitor  with  a  25x80  text  screen  is 
also  required  to  support  the  menu  system  used  by  Testbank. 


1 


2.0  INSTALLATION  PROCEDURES 


Testbank  runs  under  a  Bitnple  directory  structure.  The  software  program  itself 
resides  in  a  directory  named:  "TESTBANK.”  The  three  following  files  must 
reside  in  this  directory; 

TESTBANK.EXE  -  The  main  Testbank  program. 

TESTBANK. OVR  -  The  Testbank  overlays. 

TESTBANK. PAR  -  A  file  of  ChiWriter  setup  commands. 

In  order  to  install  the  Testbank  software,  you  should  follow  the  steps 
:ut lined  below: 

a.  From  the  root  directory,  create  the  primary  Testbank  directory  with: 

MD  TESTBANK  <enter> 

CD  TESTBANK  <enter> 

b.  New  insert  the  Testbank  Installation  disk  into  drive  A;  (or  whichever 
floppy  dri'/e  you  wish  to  use)  and  copy  over  the  required  files. 

COPi  A;  TESTBANK.  *.  •  <entet.» 

c.  To  start  up  the  Testbank  program,  type: 

TESTBANK  <enter> 

d.  Set  tne  configuration  by  selecting  the  Change  Configuration  option  on 
the  Initial  Menu.  This  option  allows  you  to  change  the  device  on  which  the 
Testbank  data  banes  will  be  stored  {"Testbank  Device")  and  the  device  where 
T'hiWritpr  resides  ("ChiWriter  Device").  Both  of  these  devices  are  defaulted 
to  the  "Cl"  drive. 

NuTE  1:  To  fully  "tilize  the  Testbank  functions,  you  will  need  a  copy  of 
ChiWriter  installed  on  your  hard  drive.  Specific  instructions  for  installing 
p’-iWritP'  are  in  Chapter  I  of  the  ChiWriter  manual. 

NOTE  2:  To  start  up  Testbank  later  on,  you  will  need  the  following: 

CD  TESTBANK  <enter> 

TESTBANK  <enter> 

Or.ee  the  Testbdi'.k  software  has  been  installed,  you  can  use  the  New  Co'  rse 
jpcion  CP  the  Initial  Menu  to  create  data  bases  for  specific  courses.  Yoi;  may 
1,30  use  t  ie  Hestore  Course  option  of  the  Misc  Menu  to  copy  in  an  existing 
course  dira  base.  Specifics  on  these  options  appear  later  in  this  manual. 
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3.0  GENERAL  INFORMATION 


Prior  to  learning  the  apecifica  of  operating  the  varioua  Teatbank  functions, 
you  will  find  it  helpful  to  understand  some  of  the  basic  concepts  used  by  the 
program. 

3.1  Screen  Layout.  In  particular,  the  Teatbank  software  uses  a  partitioned 
screen  layout  with  menus  and  function  keys  to  allow  you  to  access  the  Testbank 
capabilities.  Figure  3-1  below  illustrates  the  screen  layout. 


Figure  3-1,  Teatbank  Screen  Layout 

3.1.1.  Infomation  Line:  This  is  the  top  line  of  the  screen  and  keeps  track 
of  your  currently  selected  course,  test.  Item,  and  (when  multiple  objects  are 
queued  for  your  review)  the  current  and  total  number  of  your  responses. 

3.1.2.  Main  Bar  Menu:  Once  a  course  data  base  has  been  selected,  the  second 
line  from  the  top  of  the  display  becomes  the  Main  Menu  Bar.  This  line  allows 
you  to  access  the  three  primary  functional  menus.  These  menus  allow  you  to 
build  and  modify  tests,  maintain  items,  and  perform  miscellaneous  functions. 
The  Main  Bar  Menu  will  normally  remain  visible  to  let  you  know  your  current 
functional  area. 

3.1.3.  Pull-Down  Menus:  Each  of  the  three  primary  functions  has  its  own 
pull-down  menu  that  activates  when  you  select  the  function.  These  menus 
provide  you  access  to  the  next  level  of  functions.  Pull-down  menu  options  may 
be  selected  directly  by  hitting  the  appropriated  highlighted  letter  or  by 
arrowing  down  to  the  desired  function  and  hitting  the  ENTER  key. 

3.1.4.  Secondary  Menus:  Some  of  the  pull-down  menu  options  have  their  own 
lower  level  options  that  appear  on  a  secondary  menu.  These  options  are 
selected  in  the  same  fashion  as  the  pull-down  menu  options. 

3.1.5.  Dialog  Boxes:  When  need  to  specify  an  item  number,  file  name,  or 
ci^.her  such  information,  a  djciog  box  will  appear  with  a  place  for  you  to  enter 
the  information.  When  entering  the  information,  you  will  have  several 
standard  keys  available  to  you.  These  include  the  backspace,  delete,  home, 
end,  insert,  and  left/right  arrows. 
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3.1.6.  Forms t  When  several  items  of  information  need  to  be  specified,  a  form 
will  be  displayed  with  various  fields  indicated.  Information  in  the  fields 
can  be  specified  just  as  in  the  dialog  boxes.  In  order  to  warn  you  about 
exceeding  the  field's  size,  a  beep  will  sound  when  you  enter  a  character  for 
which  there  is  no  room.  You  may  move  between  fields  with  the  tab,  shift-tab, 
and  up/down  arrows.  The  FIO  function  key  will  always  submit  the  form  for 
verification  and  processing,  and  the  escape  key  will  always  terminate  the  form 
processing . 

3.1.7.  Lists:  At  times  you  will  need  to  make  a  selection  out  of  a  predefined 
act  of  information.  When  this  occurs,  you  will  be  presented  with  a  list  of 
the  options.  The  up  and  down  arrows,  together  with  the  page  up  and  page  down 
commands,  are  available  for  you  to  move  through  the  list.  A  carriage  return 
allows  you  to  select  an  item  from  the  list. 

3.1.8.  Submenu  Line:  In  addition  to  the  menu  system,  function  keys  are  often 
available  for  your  use.  The  second  from  the  last  line  of  the  display  is 
called  the  submenu  line  and  identifies  the  currently  available  function  keys 
and  their  functions.  Throughout  the  Testbank,  you  will  see  that  FIO  is  used 
to  execute  a  command/process ,  and  the  escap>e  key  is  standard  for  moving  back 
C'.L  of  functions. 

3.1.9.  Description  Line:  The  last  line  of  the  display  is  called  the 
description  line.  This  line  provides  you  with  a  brief  description  of  the 
currently  selected  menu  function. 

1.1.10.  Feedback  Area:  The  right  end  of  the  description  line  is  used  as  a 
feedback  area.  Whenever  a  process  executes  for  more  than  a  second  or  so,  a 
message  will  flash  in  this  area  indicating  what  is  currently  happening.  When 
a  command  is  completed,  this  area  will  either  be  cleared  or  a  completion 
message  will  appear. 

.3.2  Error  Messages  Errors  encountered  during  operation  of  the  Testbank  are 
reported  in  red  boxes  and  a  list  of  error  codes  is  provided  in  Appendix  A  to 
assist  you  in  resolving  errors. 

3  •  3 _ Function  Keys.  Once  a  function  has  been  selected  via  the  menu  system, 

most  of  the  lower  level  commands  are  accomplished  with  the  use  of  function 
keys.  The  currently  available  function  keys  are  identified  on  the  submenu 
line.  FIO  is  consistently  used  to  initiate  or  confirm  an  operation.  The 
escape  key  is  typically  used  to  get  out  of  a  function.  When  the  escape  key  is 
used  in  a  situation  that  could  result  in  the  loss  of  already  accomplished 
work,  you  will  be  prompted  to  ensure  that  this  is  appropriate. 

3.4  ChiWriter  Interface.  The  items  stored  in  the  Testbank  data  bases  are 
ChiWriter  documents.  The  Testbank  software  will  normally  display  these  items 
in  an  abbreviated  format.  This  abbreviated  format  does  not  show  all  of  the 
symbols  that  ChiWriter  provides,  but  it  is  very  fast.  In  order  to  see  the 
full  display  of  the  item  as  it  would  appear  in  a  test,  Testbank  allows  you  to 
-all  up  ChiWriter  to  display  the  item.  This  is  typically  done  with  the  F2 
function  key.  ChiWriter  is  also  used  to  finalize  a  completed  teat  document. 
Once  in  ChiWriter,  all  of  the  normal  ChiWriter  commands  are  available.  When 
,ju  exit  ChiWriter  with  an  Escape/Qult  command  sequence,  you  will  be  returned 
•'o  the  Testbank  processing.  For  further  details  on  the  ChiWriter  commands, 
see  Chapter  II  of  the  ChiWriter  manual. 
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3.5  Teat  Item  Reference  Data.  The  Testtank  uses  historical  data  to  provide 
the  reliability  and  predictability  of  tests.  Each  test  item  will  have  a 
reference  record  associated  with  it  for  every  time  the  item  has  been  used  in  a 
test.  The  data  in  the  reference  record  is  used  for  answering  queries  for  test 
items  ana  to  aid  the  developer  with  the  selection  of  appropriate  items.  In 
particular,  a  reference  record  will  include  both  primary  and  secondary 
keywords,  item  level,  item  category,  the  usage,  ease,  selectivity,  test  and 
multiple-choice  statistics,  a  breakout  of  option  selections,  and  comments. 

3.5.1.  Keywords:  Each  item  has  a  primary  and  maybe  a  secondary  keyword 

associated  with  it.  Only  the  primary  keyword  is  required  by  the  Testbank. 
These  keywords  are  very  important  since  they  will  most  likely  be  used  for 
selecting  items  for  Inclusion  in  teste.  Consistent  use  of  keywords  is 
critical.  We  do  not  want  to  use  "derivatives"  as  a  keyword  if  "derivative" 
has  already  been  used,  since  this  means  we  would  have  to  use  two  different 
queries  to  retrieve  items  that  test  the  concept  of  derivative.  In  order  to 
facilitate  the  needed  keyword  consistency,  any  time  a  keyword  is  to  be  entered 
nr  selected,  the  F  function  key  will  be  available  to  call  up  a  current  list 
of  keywords.  Furthermore,  there  is  a  Print  Keyword  List  option  available 
under  the  Item  menu.  This  option  allows  you  to  write  a  list  of  the  keywords 
to  the  printer  or  to  a  file. 

3.5.2.  Item  Level:  The  item  level  is  an  assessment  of  the  level  and  type  of 
effort  required  to  answer  the  item.  It  is  a  required  field  in  the  reference 
fonr..  There  are  four  options  for  this  assessment.  They  are  computation  (CT), 
comprehension  (CP),  application  (AP) ,  and  analysis  (AN).  These  options  are 
defined  in  detail  below  using  definitions  taken  from  E.G.  Begle's  Critical 
Variables  in  Mathematics  Education,  1979,  p  15. 

COMPUTATION  -  Items  designed  to  require  straight-forward  manipulation 
of  problem  elements  according  to  rules  the  subjects  presumably 
have  learned.  Emphasis  is  upon  performing  operations  and  not 
upon  deciding  which  operations  are  appropriate. 

COMPREHENSION  -  Items  designed  to  require  either  recall  of  con¬ 
cepts  and  generalizations  or  transformation  of  problem  elements  from 
from  mode  to  another.  Emphasis  is  upon  demonstrating  understanding 
of  concepts  and  their  relationships  and  not  upon  using  the  concepts 
to  produce  a  solution. 

APPLICATION  -  Items  designed  to  require  (1)  recall  of  relevant  know¬ 
ledge,  (2)  selection  of  appropriate  operations,  and  (3)  performance  of 
the  operations.  Items  are  of  a  routine  nature.  They  require  the 
subject  to  use  concepts  in  a  specific  context  and  in  a  way  presum¬ 
ably  already  practiced. 

ANALYSIS  -  Items  designed  to  require  a  nonroutine  application  of  concepts. 

J  .  *> .  J .  Category:  Category  refers  to  the  type  of  test  question;  the  options 
f 1 multiple  choice  (MC),  short  answer  (SA),  and  work  out  (WO).  This  is  a 
rsquiced  field. 

3  'i .  4 .  Usage:  This  field  is  used  to  indicate  the  teat  into  which  the  item 
vaa  included  to  produce  the  associated  reference  data.  The  format  of  this 
field  is  tttvsnn  where  ttt  is  the  test  name  ( 'GRl '  .  .  . 'GR5 ,  '  '  FIN '  ) ,  v  is  the 
tsat  .sieion  ( 'A' , 'B,  ' 'M' ) ,  s  is  the  semester  ('F,''S'),  and  nn  is  the  year. 
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3.5.5.  Basai  The  ease  indicates  the  percent  of  those  taking  the  test  who 
correctly  answered  che  item.  The  average  of  all  the  ease  statistics 
associated  with  an  item  (weighted  by  number  taking  each  test)  is  the  number 
used  for  satisfying  queries  for  test  items  and  for  predicting  test  statistics. 

3.5.6.  Selectivityt  The  selectivity  is  a  measure  of  the  how  well  the  item 
discriminated  between  the  students  who  mastered  the  material  and  those  who 
have  not.  This  statistic  is  calculated  by  dividing  the  population  that  took 
the  test  into  three  nearly  equal  size  groups.  One  group  contains  those  that 
did  the  best  on  the  considered  items.  Another  group  consists  of  those  with 
the  worst  scores.  These  two  group  are  of  equal  sizes.  The  third  group 
consists  of  those  remaining  in  the  middle.  For  example,  if  the  population 
taking  the  test  were  ten,  then  the  high  and  low  groups  would  each  have  three 
and  the  middle  group  would  have  four.  The  selectivity  statistic  ranges 
between  -1  and  1  and  is  calculated  with  the  formula  below: 


SEL 


(#  in  high  group  correct)  -  (#  in  low  group  correct) 


(Total  #  in  high  group) 


A  value  of  1  indicates  a  high  selectivity  where  all  in  the  high  group  answered 
correctly  and  all  in  the  low  group  answered  incorrectly.  A  value  of  -1 
indicates  that  all  in  the  high  group  got  the  item  wrong  and  the  low  group  all 
got  it  right.  With  the  exception  of  extremely  easy  or  extremely  hard 
questions,  the  selectivity  value  should  be  greater  than  0.2. 

WARNING:  The  selectivity  index  for  an  item  is  sensitive  to  the  quality  and 
number  of  the  other  items  with  which  it  was  given,  together  with  the  number  of 
students  who  took  the  test.  Few  accompanying  questions,  poor  quality 
accompanying  questions,  misgraded  accompanying  questions,  and  small  test 
populations  will  severely  degrade  the  value  of  this  statistic. 


3.5.7.  Test  and  Multiple-Choice  Statistics:  In  order  to  provide  some  insight 
into  the  context  in  which  the  item  was  used,  we  also  include  the  overall  test 
average,  standard  deviation,  and  population  size  together  with  the  same 
statistics  for  the  multiple  choice  portion  of  the  test.  The  average  and 
standard  deviation  are  given  as  percents  to  avoid  point  biases.  The 
population  sizes  are  later  used  to  weight  the  ease  indicies  associated  with  an 
item. 


3.5.8.  Selection  Break-Outs:  Also  included  in  the  reference  data  is  a  break¬ 
out  of  how  many  students  in  each  of  the  three  groups  (described  in  3.5.6) 
selected  each  of  the  multiple  choice  options.  This  is  useful  in  determining 
strong  and  weak  distractors  and  can  help  you  refine  the  item. 

3.5.9.  Comments:  There  is  room  for  4  lines  of  60  characters  for  comments 

''Out  the  item.  The  comments  could  include  things  like  how  the  item  was 

developed  (i.e.,  reasoning  behind  distractors,  etc.)  or  specifics  regarding 
its  performance  that  are  not  reflected  or  obvious  from  the  other  data. 
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4.0  START-UP  INSTRUCTIONS 


The  Teatbank  software  is  executed  by  the  following  set  of  DOS  commands  that 
could  easily  be  put  into  a  batch  file; 

CD  TESTBANK  <enter> 

TESTBANK  <enter> 

Once  initiated,  a  banner  display  is  shown  that  includes  the  copyright  and 
usage  restrictions.  Hitting  any  key  at  that  point  will  take  you  to  the  intial 
menu  (see  Figure  4-1).  The  first  item(s)  on  this  menu  is  the  course{s) 
currently  available.  When  you  select  one  of  these  courses,  you  will  be 
prompted  for  a  password  that  is  specific  to  the  course  and  then  taken  to  the 
main  bar  menu. 


HATIIMI 
I  nATtiit] 

I  HAlmiO 
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Oiiree 
Co-tf  t<|<ire 
•I  r>-i>'te  cre 
Uu  t  c 


i'Oown  •«!  i  -v)n  «»r  riA'-ute  SSC'QuIt 

f-»r  HMH141 


Figure  4-1,  Initial  Menu 

Following  are  several  other  options  available  on  the  initial  menu. 

4  ■  1  New  Course  This  option  creates  a  new  (empty)  data  base  for  a  course. 
This  data  base  will  exist  in  its  own  subdirectory  with  the  same  name  as  the 
course.  As  a  result,  course-naming  restrictions  are  the  same  as  those  for 
naming  DOS  subdirectories.  When  a  new  course  is  created,  the  initial  password 
associated  with  it  will  be  the  same  as  the  course  name.  This  may  then  be 
changed  by  an  option  on  the  Misc  menu. 

4.2  Delete  Course  This  rather  drastic  command  completely  erases  the  data  base 
associated  with  a  particular  course.  Due  to  its  serious  consequences,  after 
selecting  this  option  you  will  be  prompted  to  ensure  that  this  is  really  what 
you  want  to  do. 
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4.3  Change  Configuration  The  Testbank  software  allows  the  actual  course  data 
bases  to  exist  on  separate  disk  drives.  It  also  can  handle  ChiWriter  being  on 
a  different  disk  drive.  The  Change  Configuration  command  provides  a  submenu 
that  allows  you  to  change  either  the  Testbank  drive  and/or  the  ChiWriter  drive 
(see  Figure  4-2).  You  may  also  display  the  current  drives.  Once  changed,  the 
drive  settings  will  be  permanently  saved.  If  the  Testbank  drive  is  changed, 
the  new  drive  will  be  checked  and  the  initial  menu  changed  to  reflect  the 
newly  available  courses  on  that  drive. 


Figure  4-2,  Change  Configuration  Option  and  Submenu 
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5.0  TEST  GENERATION  AND  MAINTENANCE 


This  set  of  functions  allows  tests  to  be  created,  modified,  finalized,  and 
erased.  The  pull-down  menu  for  this  set  of  functions  consists  of  "Build 
Test,"  "List  Tests,"  "Finalize  (CHI),"  and  "Erase  Test"  (see  Figure  5-1). 
When  any  option  other  than  List  Testa  is  selected,  you  will  be  prompted  for  a 
test  name.  You  may  either  type  in  the  name  directly  or  hit  the  F5  function 
key  to  obtain  a  list  of  existing  tests  for  selection.  Once  a  test  has  been 
selected,  the  information  line  (top  line  of  display)  will  be  updated  with  that 
teat  name. 


Figure  5-1,  Test  Pull-Down  Menu 

5.1  Build  Test .  This  is  the  primary  option  that  allows  tests  to  be  created  and 
modified.  Within  this  option  you  may  also  predict  the  statistics  for  a 
created  test  and  display  the  items  included  in  the  test. 

When  you  initially  build  a  new  test,  an  item  query  form  will  be  displayed  for 
you  to  select  items  for  inclusion  into  test.  If  the  test  you  selected  already 
exists,  a  summary  of  the  test  will  be  displayed  and  a  number  of  different 
options  are  then  available  to  you. 

5.1.1.  Item  Queries:  Testbank  provides  an  item  qpjery  form  for  selecting 
items  for  inclusion  in  the  test  (see  Fig  5-2).  The  query  form  allows  any 
combination  of  the  query  parameters.  These  query  parameters  include  keyword, 
level,  category,  ease  range  (0.0  to  100.0  possible),  selectivity  range  (-1.0 
to  1-0),  and  item  number.  Items  returned  for  a  given  query  will  have  to 
satisfy  all  of  the  parameters  that  you  specify.  The  one  exception  to  this 
rule  is  when  you  specify  item  number.  In  this  case,  the  specified  item  will 
be  returned  even  if  it  does  not  satisfy  the  other  parameters.  The  F5  function 
key  will  provide  you  with  selection  lists  when  specifying  either  the  keyword 
parameter  or  the  item  number. 
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Figure  5-2,  Item  Query  Form 

5.1.2.  Query  Responses:  Once  an  error-free  query  has  been  submitted  via  the 
FIO  function  key,  a  search  of  the  data  base  will  be  accomplished  and  a  list  of 
responses  built.  When  the  search  is  completed,  the  first  response  item  will 
be  displayed  and  the  right  aide  of  the  information  line  (top  line)  will  report 
the  total  number  of  responses  (see  Figure  5-3).  The  F3  and  F4  function  keys 
will  allow  you  to  move  through  the  list  of  response  items.  The  F2  key  will 
redisplay  your  current  item  using  ChiWriter.  F6  will  display  the  reference 
data  for  your  current  item,  and  F7  will  display  any  earlier  versions  of  the 
item  that  may  have  been  used  in  the  past  and  were  revised.  The  escape  key 
will  take  you  back  to  the  query  form  and  get  you  out  of  the  response  review. 
The  FIO  function  key  will  select  your  currently  displayed  item  for  inclusion 
into  the  teat  and  take  you  back  to  the  test  summary.  When  an  item  is 
selected,  you  will  be  prompted  for  its  point  value;  you  must  then  specify  a 
positive  integer  value  (point  value  may  be  changed  later).  If  you  attempt  to 
include  an  item  into  the  test  that  is  already  in  the  test,  you  will  be  shown 
an  error  message  and  disallowed  the  second  inclusion. 


Figure  5-3,  Response  Review  Item  Display 

5.1.3.  Test  Sujsmary  Index:  Once  your  current  test  has  at  least  one  item,  a 
test  summary  index  will  be  displayed  (see  Figure  5-4).  This  index  display 
provides  the  sequence  number,  point  value,  item  nunnber,  primary  keyword, 
category,  level,  ease,  and  selectivity  for  each  item  in  the  test.  (The  ease 
and  selectivity  are  actually  accumulated  figures  derived  from  all  uses  of  the 
i-tem)  .  The  up  and  down  arrows  can  be  used  to  select  items  in  the  test  for 
display,  replacement,  or  deletion.  .If  there  are  more  than  16  items  in  your 
test,  you  can  use  the  page  up  and  page  down  keys  to  access  the  other  items. 
The  test  summary  index  is  intended  to  provide  a  good  summary  for  the  test 
developer  to  use  in  assessing  test  content  coverage. 
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Figure  5-4,  Teat  Summary  Index 


The  Test  Summary  Index  provides  acceaa  to  a  number  of  functions  accessible  via 
the  following  function  keys: 

a.  Fl-Display  Item:  This  function  will  put  up  a  fast  display  of  the  item 
currently  selected  on  the  Teat  Summary  Index.  Once  displayed,  you  may  use  F2 
to  fully  display  it  with  the  ChiWriter  word  processor  or  use  F6  to  display  the 
reference  data  associated  with  this  item. 


b.  F4-Replace  Item:  This  function  allows  you  to  replace  the  currently 

selected  item.  The  query  form  used  to  find  the  original  item  will  be  provided 
for  you  to  update  and  submit  to  find  the  replacement  item. 

c.  F5-Predict  Statistics;  When  executed,  this  function  provides  an 
estimate  of  the  test's  average  and  standard  deviation  based  on  the  accumulated 
reference  data  for  each  item  in  the  test  that  has  ease  and  selectivity  data. 
If  one  or  more  items  in  the  test  do  not  have  the  required  reference  data,  they 
will  not  be  included  in  the  estimates.  See  Figure  5-5  for  an  example  of  the 
predicted  statistics.  The  lower  bound  for  standard  deviation  is  based  on  a 
variance  prediction  that  assumes  independent  test  items.  An  actual  standard 
deviation  less  than  the  lower  bound  indicates  that  there  are  negative 
dependencies  between  the  test  items;  i.e.,  getting  one  item  correct  means  you 
^re  more  likely  to  get  another  problem  wrong.  This  would  be  indicative  of  a 
poorly  constructed  test. 

d.  F6-Add  Item:  The  add  item  function  brings  up  the  item  query  form  to 
allow  you  to  find  and  select  another  item  for  inclusion  into  the  test. 

e.  F7-Reorder  Teat:  This  function  translates  your  current  page  of  the 

■Ht:  Summary  Index  into  a  form  (see  Figure  5-6)  with  the  last  two  fields  on 

la'h  line  available  for  you  to  modify  the  sequence  number  and  point  value  for 
item.  Sequence  numbers  can  be  signed  integers,  and  point  values  must  be 
pisitLve  integers  less  than  1000.  Once  submitted,  all  the  items  in  the  test 
w.il  be  sorted  by  sequence  number  and  resequenced  starting  at  one.  With  this 
mechanism,  it  is  easy  to  put  an  item  at  the  front  of  the  test  by  assigning  it 
a  large  negative  sequence  number.  NOTE:  Changing  point  values  can  affect  the 
predicted  statistics. 

f.  F8-Delete  Item;  Hitting  the  F8  key  will  cause  the  currently  selected 
.•'  •m  in  the  Test  Summary  Index  to  be  removed  from  the  test.  Executing  this 

:  .Tjnarid  will  result  in  a  verify  prompt  to  help  ensure  the  wrong  item  is  not 
e .eted . 
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g.  F9-Print  Index:  This  function  allows  you  to  either  print  or  write  to 
a  file  the  Test  Sutwnary  Index  with  the  predicted  statistics  attached.  When 
you  execute  this  command,  you  will  be  prompted  for  a  "PRINTER/FILE  NAME"  and 
the  default  value  will  show  up  as  "LPTl"  which  is  the  standard  printer.  You 
may  either  hit  <enter>  to  print  the  index  on  LPTl,  or  you  may  type  in  a  file 
name  or  other  printer  device.  Anytime  during  the  print  procedure  you  can  hit 
the  escape  key  to  terminate  the  printing  and  return  to  the  Test  Summary  Index. 

h.  FlO-Save  Test:  As  usual,  FIO  allows  you  retain  your  work.  Hitting 
FIO  will  save  the  Teat  Summary  Index  to  disk  and  then  let  you  escape  or 
continue  working  on  the  teat.  Once  saved,  the  test  may  be  called  up  again 
later  by  executing  the  Build  Test  option  or  it  may  be  finalized  into  a 
ChiWriter  document  with  the  "Finalize  (CHI)"  option.  We  recommend  hitting  FIO 
frequently  during  test  development  so  as  to  not  lose  your  work. 

i.  Escape  Key:  The  escape  key  returns  you  to  the  Test  menu.  If  you  have 
created  a  teat  and  not  saved  it,  a  prompt  will  appear  to  ensure  that  you  wish 
to  abandon  your  work  without  saving  it. 


1  TtSTSAN*  ( 1 , 

0)  Cowrssi  nartiiai  T**#'  •  CSl-A  ttO  Itoe; 

Test 

Itae 

0u!» 

tut  14  teat 

:.ist  tstts 

r tox  1  ( (s  (Oil t 

Crate  test 

MfOICTfO  T^^r  STATIST  iC.t 

•seed 

on  IS  '>f  It  lt*ee  (  Ijeo  of  )ooo  ptai 

tredlcted  A«o  at  Olt 

TredlcT*'!  Roxfwi  std  0e«-  ii  itt 

Prsdtc 

rtd  SIO  Oev:  It  t-’% 

- - 

t  '•< 

•  tar  ur  o 

•  Ttsr  ^ 

. 

1-1  « 

or  r  •«  lav 'eort  1  r  .  aalariiv*  >*«r 

Figure  5-5,  Testbank  Predicted  Statistics 
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Figure  5-6,  Test  Summary  Index  Reorder  Form 

S.2  List  Tests.  This  Test  menu  option  simply  displays  a  list  of  the  current 
tests.  If  there  are  more  tests  than  can  fit  on  the  display,  you  can  use  the 
page-up  and  page-down  commands  to  see  the  rest  of  the  tests. 
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5.3  Finalize  (CHI).  Selecting  this  Test  menu  option  prompts  you  for  a  test 
name.  The  F5  function  key  will  provide  a  list  of  current  tests.  Once  a  test 
has  been  selected,  Testbank  will  merge  all  of  the  items  in  the  test  into  a 
single  ChiWriter  document  and  bring  the  teat  up  for  you  in  the  ChiWriter  word 
processor.  Within  ChiWriter,  you  can  adjust  spacing  between  items  and  insert 
page  breaks  as  necessary.  You  may  also  add  headers,  footers,  and  cover  sheets 
as  required.  Specifics  on  editing  with  ChiWriter  may  be  found  in  Chapter  II 
of  the  ChiWriter  manual. 

Since  ChiWriter  will  be  run  while  the  Testbank  program  is  still  in  memory,  the 
size  of  the  document  that  ChiWriter  can  handle  will  be  limited.  You  will  be 
notified  of  your  test  exceeds  this  limit  and  be  given  the  opportunity  to  copy 
your  test  document  to  another  file  that  you  may  later  edit  with  ChiWriter 
after  you  have  exited  Testbank  and  more  memory  is  available. 

WARNINO:  Once  you  have  edited  the  finalized  test,  you  must  write  it  to  a  file 

with  a  new  name.  The  file  created  by  Testbank  during  the  merge  process  will 
be  automatically  deleted  for  security  reasons.  Thus  you  will  need  to  execute 
an  Eecape/Write/Change  Name  command  seq[uence  followed  by  an  Escape/Write/ 
Document  to  preserve  your  ChiWriter  edits. 

OTHER  CONCERNS:  During  the  merge  process,  Testbank  will  prefix  each  test  iterr. 

with  the  point  value  and  sequence  number  for  that  item.  These  will  be  put  in 

front  of  the  first  nonempty  main  line  of  the  item's  text.  When  items  include 
extensive  use  of  superscript  lines,  the  point  value  and  sequence  number  niay 
not  be  put  exactly  where  you  like  them  but  may  be  editted  with  ChiWriter.  If 
the  first  text  line  of  an  item  is  especially  long,  prefixing  the  point  value 
and  sequence  number  may  cause  it  to  exceed  line  size  and  be  truncated.  If 

this  occurs,  you  will  be  warned  as  ChiWriter  is  started  up. 

5.4  Erase  Test.  The  Erase  Test  option  also  prompts  you  for  a  test  name  and 
provides  a  list  of  existing  teats  via  the  F5  function  key.  Once  a  test  is 
selected,  Testbank  will  verify  that  you  wish  to  delete  it  to  help  preclude 
errors . 
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6.0  ITEM  MAINTENANCE 


The  Item  Maintenance  menu  is  typically  used  by  the  Testbank  Maintainera  and 
Teatbank  Adminatratora  to  build  and  ensure  the  integrity  of  the  courae  data 
baaea.  However,  aome  of  these  functions  are  also  used  by  Teat  Developers  to 
insert  new  items  into  the  data  base  that  they  may  need  to  build  a  test.  The 
Item  Maintenance  functions  are  available  via  the  Item  pull-down  menu  (see 
Figure  6-1 ) . 


Figure  6-1,  Item  Pull-Down  Menu 

6 . 1  Add  Item .  Items  can  be  added  to  a  course's  data  base  in  one  of  three  ways 
( see  Figure  6-2 )  . 


Figure  6-2,  Add  Item  Secondary  Menu 


6.1.1.  File  Add  Option:  The  File  Add  option  copies  an  item  that  currently 
exists  as  a  separate  ChiWriter  document  file.  Such  files  can  be  created  with 
the  procedure  defined  in  Appendix  B.  The  procedure  involves  taking  a 
ChiWriter  version  of  a  test  and  marking/writing  each  item  into  a  separate 
file.  When  executing  this  function,  you  will  be  prompted  for  a  file  name  and 
then  Testbank  will  display  the  new  item  for  you.  At  this  point  you  may  use 
FIO  to  save  the  item,  escape  to  abort  the  item,  or  F2  to  fully  display  the 
item  with  ChiWriter.  The  item  will  not  be  put  into  the  data  base  unless  the 
FIO  key  is  depressed.  If  you  do  save  the  item,  the  Reference  Data  Form  will 
be  provided  for  you  to  insert  the  reference  data  for  the  item. 

6.1.2.  Direct  Add  Option:  Another  way  to  add  items  to  the  data  base  is  via 
the  Direct  Add  option.  This  functions  calls  up  ChiWriter  from  Teatbank  and 
allows  you  to  create  the  new  item  directly  from  ChiWriter.  When  using  this 
function,  you  must  remember  to  execute  the  Escape/Wr ite/Document  ChiWriter 
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command  sequence  prior  to  exiting  ChiWriter.  Details  on 
commands  are  available  in  Chapter  II  of  the  ChiWriter  manual, 
has  been  created  and  you  have  exited  ChiWriter,  Testbank 
confirmation  that  you  wish  to  save  the  item  in  the  data 
confirmation,  the  Reference  Data  Form  will  be  displayed  for 
reference  data  for  the  item. 


the  ChiWriter 
Once  the  item 
will  ask  for 
base.  After 
you  to  insert 


6.1.3.  Revise  Old  Option:  The  third  way  to  add  an  item  to  the  data  base  is 
to  call  up  an  existing  item  and  revise  it  using  ChiWriter.  This  Revise  Old 
option  will  not  change  the  original  item.  The  procedure  associated  with  this 
option  is  very  similar  to  the  Direct  Add  option  except  that  you  will  be 
prompted  for  the  e.xisting  item  to  revise.  At  this  prompt,  you  may  use  the  F5 
function  key  to  obtain  a  list  of  existing  items  from  which  to  select. 
Reference  data  from  the  old  item  will  NOT  be  carried  forward. 


6.2  Modify  Item.  The  Modify  Item  function  is  used  to  correct  an  existing  item 
when  its  current  version  was  poor  or  contained  errors.  The  function  allows 
you  to  modify  the  item  using  ChiWriter;  both  the  new  and  old  versions  will  be 
maintained  in  the  data  base.  However,  only  the  new  version  will  be  available 
for  inclusion  into  tests.  When  executing  this  function,  you  will  be  prompted 
for  the  item  number  of  the  item  to  be  modified.  You  may  use  the  F5  function 
key  at  this  point  to  obtain  a  list  of  the  existing  items.  ChiWriter  will  then 
be  called  up  with  the  item  for  you  to  make  the  necessary  modifications. 
Details  on  the  ChiWriter  commands  are  available  in  Chapter  II  of  the  ChiWriter 
manual.  Remember  to  execute  the  ChiWriter  command  sequence 
Eacape/Wr ite/Document  prior  to  your  exit  from  ChiWriter.  Upon  exiting 
ChiWriter,  Testbank  will  ask  you  to  confirm  the  modification  to  the  item.  FIO 
will  confirm  the  change  and  escape  will  abort  it.  If  confirmed,  you  will  be 
asked  if  you  wish  to  enter  reference  data  for  the  item.  If  so,  you  will  be 
provided  a  Reference  Data  Form. 

NOTE:  There  is  a  third  (hidden)  response  option  to  the  confirmation  prompt. 
If  you  resond  with  a  Shift-Fl  at  the  confirmation  prompt,  the  old  version  of 
the  item  will  be  overwritten  with  the  modifications  that  you  made.  This 
feature  is  primarily  provided  to  allow  the  Testbank  Administrator  to  correct 
items  that  are  already  in  the  data  base. 

i .  3  update  Reference  Data.  There  are  three  options  available  for  modifying 
and  updating  the  reference  data  associated  with  an  item  (see  Figure  6-3). 
Once  the  reference  data  for  an  item  has  been  called  up  via  any  of  the  three 
options,  it  may  be  reviewed,  modified,  extended,  and/or  deleted  via  a  standard 
aet  of  functions. 


I  r 
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Figure  6-3,  Reference  Data  Update  Options 
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6.3.1.  General  Reference  Data  Updates:  Regardless  of  the  option  used  to  call 
up  the  reference  data  for  an  item,  the  same  functions  are  available  for 
actually  updating  the  item  (see  Figure  6-4).  FI  allows  you  to  redisplay  the 
actual  item.  Since  an  item  will  have  a  separate  reference  form  for  each  time 
it  was  used,  F3  is  available  to  move  through  these  forms.  F4  will  allow  you 
to  modify  the  currently  displayed  form.  To  add  a  new  reference  data  form  for 
an  item,  hit  F6  and  a  new  form  will  be  presented  for  you  to  complete  and 
include  with  the  item.  F8  can  be  used  to  delete  the  currently  displayed 
reference  form.  Finally,  FIO  will  permanently  record  all  of  the  changes  that 
you  have  made,  and  escape  will  abort  any  changes. 


Figure  6-4,  Reference  Data  Update  Form 

6.3.2.  Individual  Itee  Reference  Data  Update:  The  first  option  available  for 
calling  up  the  reference  data  for  an  item  is  to  do  it  directly  by  item  number. 
Executing  this  option  prompts  you  for  an  item  number.  The  F5  key  is  available 
for  obtaining  a  list  of  the  current  items  in  the  data  base.  Once  an  item  has 
been  selected,  its  reference  data  is  present  for  update. 

6.3.3.  Teat  Items  Reference  Data  Update:  To  assist  with  the  updating  of 
reference  data  for  all  items  used  in  a  particular  test,  the  Test  Items  option 
is  provided.  This  option  displays  a  list  of  the  existing  tests  from  which  you 
can  select.  Once  a  test  is  selected,  a  second  list  of  all  the  item  numbers 
used  in  that  test  is  provided  for  your  use  in  selecting  items  for  updates.  As 
each  item  is  updated,  you  will  return  to  this  list  of  item  numbers. 

6.3.4.  Missing  Reference  Data  Update:  In  order  to  aid  in  the  preservation  of 
data  base  integrity,  the  Missing  Ref  Data  option  is  provided.  This  option 
causes  the  entire  data  base  to  be  scanned  for  items  that  do  not  have 
associated  reference  data.  Any  such  items  are  then  presented  in  a  list  for 
you  to  select  and  update. 

6.4  Print  ^unctions.  There  are  four  print  options  available  for  obtaining 
hardcopies  of  data  base  information  (see  Figure  6-5).  Each  of  these  options 
allows  the  information  provided  to  be  printed  or  written  to  a  file  for  later 
display  or  printing. 
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Figure  6-5,  Teatbank  Print  Functions 

6.4.1.  Item  Print:  The  Item  Print  function  prompts  for  an  item  number 
(allowing  F5  for  a  list  of  items)  and  then  calls  up  ChiWriter  with  a  copy  of 
the  item  t.h-t  rr.a^  then  be  printed  from  ChiWriter. 

6.4.2.  Reference  Data  Print:  This  function  again  prompts  for  an  item  number 
( F5  also  available  for  listing  the  items)  and  then  prints  out  all  of  the 
reference  data  associated  with  the  selected  item. 

6.4.3.  Index  Print:  The  Index  Print  option  provides  a  summary  list  of  all  of 
the  items  in  the  current  course's  data  base. 

6.4.4.  Keyword  Print:  The  consistent  use  of  keywords  is  critical  to  the 
efficient  use  of  the  Teatbank  data  bases.  The  Keyword  Print  option  provides  a 
list  of  the  current  keywords  for  reference  when  adding  new  items. 

6,5  Del ete  Item .  The  Delete  Item  function  prompts  for  an  item  number  (F5 
available  for  a  list  of  items)  and  then  deletes  the  selected  item.  A 
confirmation  prompt  is  provided  to  preclude  mistakes.  Once  an  item  has  been 
deleted,  its  number  will  not  be  used  again.  Furthermore,  if  the  deleted  item 
was  included  in  a  test,  you  will  not  be  able  to  regenerate  that  entire  test. 

,6_^6 _ Requirement  for  Back  Ups.  Every  time  any  of  the  Item  Pull-Down  menu 

functions  are  used,  you  should  generate  a  back  up  of  your  work  with  the  Back 
Up  Course  command  available  under  the  Misc  menu  and  described  in  the  next 
chapter . 
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0  misch:li.anfous  f-nctions 

T>_>3tbar:)<.  prcvidfJB  threa  miscellaneous  functions  to  assist  with  the 
adiT  in  iat  rat  ion  and  maintenance  of  the  test  data  bases  (see  Figure  7-1): 


I 


•••VI  Ml  r 


Figure  7-1,  Hisc  Full-Down  Menu 

'  '  B  -  k  'J  o Cgurj;e_.  The  Back  Up  Course  function  copies  all  of  the  item, 

reference,  and  test  files  from  the  current  course  data  base  to  floppy 
d  I  a,<et-e  (  s )  .  Any  teetbank  information  already  on  the  target  disks  will  be 
autcmut  1.  val  ly  deleted  prior  to  the  file  transfer.  However,  the  back  up 
rrocesa  will  go  much  faster  if  you  delete  all  the  files  from  the  di9kette(3) 
v  tn  "de'.,  "  before  you  use  them  for  the  back  up  process.  During  the  back 

..p  prrjce.is,  yc  u  will  be  prompted  for  a  destination  disk  drive  and  provided 
w.th  feedijs-.K  as  the  process  progresses.  This  feedback  comes  in  the  form  of 
the  number  of  f’les  copied.  There  are  two  files  for  each  test  item  and  one 
fir  each  test.  if  the  floppy  diskette  becomes  full  before  all  the  items  have 
teen  baci'e'l  up,  you  will  be  prompted  for  a  new  diskette.  Anytime  during  the 
ba.-k  up  process  you  may  hit  the  escape  key  to  abort  it. 

'  ■  t_v.?®.-.Cou_r ce The  sister  function  to  Back  Up  Course  ia  Restore  Cou  se. 

■"r.e  Ro3tor>a  Cjur''e  function  clears  your  currently  active  course  data  base  and 
reclaof's  it  with  ♦‘n-'  data  base  contained  on  back  up  disketteis).  Since  the 

curret'.t  data  .'.'-.so  rn  destroyed,  you  will  be  prompted  to  ensure  that  you  wish 

pursue  tr.is  op*'i.un.  The  files  of  the  data  base  will  be  copied  in  a  fashion 
vo’ry  s.mil  ar  *  ■;  trie  back  up  process,  and  the  restore  operation  may  be 

terminated  at  any  time  with  the  escape  key.  Wlien  a  lack  up  diskette  has  been 

u.mpletely  'u'c.cl,  you  will  be  asxed  if  thev-e  are  a.iy  ot.her  back  up  diskettes 
to  be  used. 

;  3__Ch_a_n£e _ Password .  Each  course  data  base  will  have  a  specific  password 

associated  wi-n  it.  The  password  is  required  to  restrict  access  to  the  test 
Items  and  maintain  the  integrity  of  the  data  base.  When  changing  a  password, 
y tu  will  be  prompted  for  the  old  password  and  then  prompted  twice  for  the  new 
password.  Tf  all  three  of  your  inputs  verify,  the  pa.saword  will  be  changed. 
If  any  one  of  your  inputs  are  not  correct,  the  old  password  will  remain  in 
p  f  5 ec  *: 
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8.0  TESTBANK  ADMINISTRATOR  FUNCTIONS 


One  of  the  Teetbank  Administrator ' b  primary  duties  io  to  maintain  the 
integrity  of  the  course  data  bases.  There  are  a  number  of  integrity  concerns 
to  be  dealt  with,  and  this  section  describes  them.  Other  areas  of 
responsibility  of  the  Testbank  Administrator  include  the  maintenance  of  back 
ups  and  passwords. 

8.1  Item  Integrity.  Item  integrity  concerns  include  the  format  of  the  items 
and  the  elimination  of  duplicate  items.  The  Testbank  Administrator  should 
conduct  periodic  spot  reviews  of  the  course  data  bases  to  ensure  that  newly 
included  items  are  of  a  proper  format  and  duplicate  items  do  not  exist.  An 
easy  way  to  review  items  is  to  create  a  dummy  test  using  the  Build  Test  option 
on  the  Test  menu.  Use  the  query  form  to  call  up  items  and  check  their  format. 
If  an  item  with  an  improper  format  exists,  call  it  up  with  the  Modify  Item 
option  of  the  Item  menu.  Make  the  necessary  corrections,  execute  the 
Escape/Write/Document,  and  then  Escape/Quit  ChiWriter  command  sequences.  Once 
you  are  back  in  Testbank,  respond  to  the  save  confirmation  prompt  with  a 
Shrft-Fl  (not  shown  on  the  display).  your  revised  version  of  the  item  will 
then  replace  the  errored  one. 

Things  to  look  for  when  reviewing  an  item: 

-  Removal  of  point  values  and  sequence  numbers. 

--  First  main  line  of  the  item  should  start  at  the  left  margin.  There 
should  be  at  least  ten  blanks  at  the  end  of  that  Vine  and  before  the  right 
margin.  This  is  required  so  that  sequence  nr.u.>ers  and  point  values  may  be 
added  to  items  when  they  are  included  into  tests. 

-  Absence  of  unusual  and/or  control  characters  in  the  text  of  the  item. 

•  Absence  of  page  breaks  in  the  item. 

-  A  couple  of  blank  lines  at  the  beginning  and  ending  of  each  item. 

The  easiest  may  to  look  for  duplicates  is  to  put  your  cursor  on  the  Item 
Number  field  of  the  Item  Query  form  (accessed  through  the  Build  Test  option) 
and  then  hit  F5.  You  will  be  provided  a  list  of  the  current  data  base  items. 
Look  for  items  near  each  other  that  have  identical  information,  and  call  them 
up  using  the  query  form.  Check  to  see  if  they  are  duplicates.  If  duplicate 
items  are  found,  they  may  be  easily  deleted  using  the  Delete  Item  option  under 
the  Item  menu. 

Finally,  and  perhaps  most  importantly,  the  Testbank  Administrator  should 
rnview  the  quality  of  items  being  put  into  the  course  data  bases.  Flawed 
Lter.ia,  such  as  those  with  multiple  correct  answers,  should  not  be  included  in 
•.he  data  bases.  Other  items  of  questionable  quality  should  either  not  be 
included  or  annotated  with  appropriate  comments  in  the  Comments  field  of  the 
Reference  Data  Forms. 

8.2  Reference  Data  Integrity.  Another  critical  area  of  integrity  concerns 
involves  the  reference  data  included  with  the  items.  Spotchecks  of  the 
reference  data  can  be  accomplished  in  several  ways.  Items  with  their 
reference  data  may  be  called  up  via  queries  through  the  Build  Test  option, 
likewise,  the  reference  data  for  items  may  be  called  up  directly  via  the 
Update  Ref  Data  option  of  the  Item  menu. 
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When  reviewing  reference  dat»,  you  ahoild  look  for  the  fcllowing: 

-  Consistent  use  of  key-words  (printing  ou».  a  list  of  the  keywords  will 
also  help  you  with  this  task). 

Appropriate  c  1  ase  i  f  icat  ion  of  an  item's  levf?l, 

-  Correct  specification  of  item  statistics  as  percents. 

-  Correct  .-rdering  of  response  breakdowns  for  multiple-choice  questions. 

-  Appropria-e  and  meaningful  comments. 

other  areas  to  check  regarding  reference  data  include  checking  that  all  items 
have  associat'd  reference  data  and  that  those  items  included  in  already-given 
tests  are  updated  with  the  results  of  the  test.  Both  of  these  concerns  may  be 
handled  via  the  appropriate  secondary  menu  options  under  the  Update  Ref  Data 

o  p  t  i  o  ri . 


? '  kP  h  ?  D  ailS-i of  Back  Ups.  In  order  to  ensure  that  valuable  data  is  not 

. :3t  due  to  hardware  or  operator  errors,  it  is  necessary  to  maintain  proper 
backups  of  the  course  data  bases.  The  Testbank  Administrator  should  monitor 
rhe  Testbank  Mair.tainera  to  ensure  back  ups  are  performed  after  each  session 
in  vjhich  they  work  on  a  course.  The  Testbank  Administrator  should  also 
periodically  make  a  second  set  of  backups  and  keep  them  at  a  different 
faciiit-y.  This  will  preclude  losses  due  to  major  facility  problems. 

8.4  Password  Maintenance.  When  a  course  data  base  is  initially  created,  its 
password  is  initialized  to  the  course  name.  This  password  may  then  be  changed 
via  the  Miac  menu  .‘:>y  anyone  knowing  the  current  password.  If  for  some  reason 
the  password  for  a  course  is  lost  or  forgotten,  the  Testbank  Administrator  can 
use  the  SAVEME  program  to  reset  the  password  to  "Testbank.”  To  use  the  SAVEME 
program,  simply  change  to  the  subdirectory  for  the  course  and  run  SAVEME.  For 
example,  i,  f  Testbank  were  on  my  "C"  drive  and  my  course  was  i  ..h  141,  I  would 
■vecute  'he  fcl'cuirig  rcmrranda; 

C  ••  <  e  n  t  e  r  > 
cd  testbank  <enter> 

'd  Ma  '  h  1  4  1  <Piif  er  > 
ru".  A  :SAVKME  <enter> 
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9.0  IMPLEMENTATION  SPECIFICS 


At  times,  it  may  be  helpful  to  know  a  bit  more  information  about  the  Testbank 
software  beyond  what  is  provided  in  Chapter  2.  This  section  provides  some  of 
this  information.  For  additional  details,  consult  the  Testbank  Software 
Maintainer's  Guide  and/or  the  Testbank  Software  Source  Listing. 

9.1  Directory  Structure.  The  Testbank  software  utilizes  a  simple  directory 
structure  to  maintain  separate  course  data  bases  and  isolate  data  from  the 
programs.  The  top  of  the  structure  consists  of  a  TESTBANK  directory  that 
resides  directly  under  the  disk's  root  directory.  The  TESTBANK  directory  will 
contain  the  actual  Testbank  software  and  configuration  files.  Immediately 
under  the  TESTBANK  directory  will  be  separate  subdirectories  for  each  of  the 
course  data  bases.  Figure  9-1  illustrates  the  structure. 


C; 


{Root  directory 
{Program  directory} 


\TESTBANK\MATH141  C ; \TESTBANK\MATH1 42  {Course 


directories} 


Figure  9-1,  Testbank  Directory  Structure 

9.2  File  Naming.  Within  each  course  directory,  the  contents  of  that  data  base 
are  contained  in  three  types  of  files.  These  separate  types  of  files  include 
one  type  for  the  items  themselves,  one  type  for  the  reference  data,  and  one 
type  for  the  test  summaries.  Notice  that  built  tests  exist  only  in  the 
summary  form  from  which  ChiWriter  versions  are  built.  This  makes  it  necessary 
for  the  Test  Developer  to  write  his/her  test  to  a  separate  file  during  the 
Finalize  (CHI)  option  on  the  Test  menu.  The  names  for  the  three  types  of 
files  follow  the  conventions  below: 

a.  Item  Files:  TInnnnnv.CHI  where  nnnnn  is  the  item  number  padded  to  the 
left  with  zeroes  and  v  is  the  version.  The  latest  version  will  be  "A"  and  it 
will  be  the  only  version  that  can  be  incorporated  into  tests.  Other  versions 
Will  be  named  "B, "  "C,"...  and  are  available  only  for  historical  reference 
purposes.  These  files  are  actually  encrypted  read-only  versions  of  the 
^hiWriter  format  items. 

b.  Reference  Files:  TInnnnnv.REF  where  nnnnn  is  the  item  number  padded  to 
the  left  with  zeroes  and  v  is  the  version.  These  files  are  of  a  special 

estbank  software  defined  format  and  include  one  record  of  data  for  each  time 
‘he  Item  was  used.  When  the  item  is  used  more  than  once,  a  summary  record 
will  be  included  at  the  end  of  the  file  with  the  accumulated  statistics  for 
the  item. 

c.  Test  Summary  Files:  tttvsyy.TST  where  ttt  is  the  test  ID,  v  is  the 
version  (A,  B,  or  M) ,  s  is  the  semester,  and  yy  is  the  year.  An  example  would 
oe  GR1AF90.TST.  These  files  are  also  of  a  Testbank  software-defined  format 
and  include  one  record  for  each  item  included  into  the  test.  This  record  will 
include  the  item  number,  sequence  number,  point  value,  and  query  parauneters 
used  to  find  the  item.  This  file  is  used  as  a  table  of  contents  for  building 
the  actual  ChiWriter  version  of  the  test. 
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9.3  Back  Up  and  Reatore  Functions.  The  back  up  an  i  restore  functions  built 
into  Testbank  under  the  Misc  menu  are  based  on  copying  individual  files. 
While  simple  and  easy  to  understand,  this  is  not  the  most  efficient  means 
available  since  DOS  limits  diskettes  to  112  files.  A  more  efficient  backup 
mechanism  exists  via  the  DOS  BACKUP  and  RESTORE  commands.  These  commands  m.--; 
be  used  to  back  up  and  restore  the  contents  of  the  course  subdirec t cr i en  in  a 
more  efficient  manner. 
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Appandix  A:  Error  Codas 


These  error  codes  are  the  standard  DOS  error  codes  that  may  also  be  easily 
found  in  the  DOS  and  Turbo  Pascal  manuals. 

DOS  ERRORS; 

2  -  File  not  found. 

3  -  Path  not  found. 

4  -  Too  many  files  open. 

5  -  File  access  denied. 

6  -  Invalid  file  handle. 

12  -  Invalid  file  access  mode. 

15  -  Invalid  drive  number. 

16  -  Cannot  remove  current  directory. 

17  -  Cannot  rename  across  drives. 

INPUT/OUTPUT  ERRORS: 

100  -  Disk  read  error. 

101  -  Disk  write  error. 

102  -  File  not  assigned. 

103  -  File  not  open. 

104  -  File  not  open  for  input. 

105  -  File  not  open  for  output. 

106  -  Invalid  numeric  format. 

CRITICAL  ERRORS: 

150  -  Disk  is  write-protected. 

151  -  Unknown  unit. 

152  -  Drive  not  ready. 

153  -  Unknown  command. 

154  -  CRC  error  in  data. 

155  -  Bad  drive  request  structure  length. 

156  -  Disk  seek  error. 

157  -  Unknown  media  type, 
iih  -  Sector  not  found. 

159  -  Printer  out  of  paper. 

160  -  Device  write  fault. 

161  -  Device  read  fault. 

162  -  Hardware  failure. 

FATAL  ERRORS: 

200  -  DiviBicn  by  zerc . 

211  -  Range  check  error. 

202  -  Stack  overflow  error. 

203  -  Heap  overflow  error. 

204  -  Invalid  pointer  operation. 

205  -  Floating  point  overflow. 

206  -  Floating  point  underflow. 

207  -  Invalid  floating  point  operation. 
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Appendix  B:  Itea  Preparation  with  ChiWritar 


The  easiest  way  to  get  items  for  course  data  bases  is  by  taking  them  from 
existing  tests  created  with  the  ChiWritar  word  processor.  This  is  done  by 
using  ChiWriter  to  break  the  teat  into  items  and  write  the  items  to  separate 
files.  This  procedure  is  designed  to  guide  this  break  out  process  and  create 
resulting  files  that  may  be  included  into  the  Testbank  data  bases  without 
further  modification. 

1.  Start  up  ChiWriter  with  the  test  document  (assume  the  teat  file  to  be 
called:  "TEST. CHI"  for  purposes  of  illustration). 

C:  <enter> 

CD  CHI  <enter> 

CW  TEST. CHI  <enter> 

2.  Move  down  to  the  start  of  the  first  item  in  the  test  and  prepare  it  for 
marking.  The  following  critical  checks  must  be  done  at  this  point: 

a.  Ensure  there  are  at  least  two  blank  lines  (with  no  page  breaks)  before 
and  after  the  item. 

b.  If  the  item  has  a  secpjence  number  and/or  point  value  with  it,  delete 
the  sequence  number  and  point  value.  This  should  leave  some  spaces  at  the  end 
of  this  first  line.  Ensure  that  there  are  at  least  10  such  spaces.  (These 
will  be  needed  when  the  item  is  included  into  a  later  teat  so  that  the  new 
sequence  number  and  point  value  may  be  put  at  the  start  of  the  first  line 
Without  the  Line  growing  too  long  and  being  truncated.) 

c.  If  the  first  line  of  the  item  contains  a  lot  of  super/aubscr ipts ,  you 
should  consider  creating  a  new  first  line  prior  to  the  current  one.  This  is 
ueeded  because  the  Testbank  software  will  prefix  the  point  value  and  sequence 
number  for  an  item  at  the  beginning  of  the  first  main  line  of  an  item.  If  the 
item  has  a  lot  of  supers/subscr ipts  on  that  line,  the  point  value  and  sequence 
number  may  be  put  in  an  inappropriate  place. 

3.  Once  step  2  is  completed,  the  item  may  be  marked  and  written  to  a  file. 
The  following  steps  should  then  be  observed: 

a.  Place  the  cursor  at  the  begining  of  the  two  blank  lines  that  precede 
the  item. 

b.  Execute  an  Escape/Mark. 

c.  Move  the  cursor  to  the  end  of  the  second  line  after  the  item  and 
execute  an  Escape/Write. 

d.  When  prompted  for  a  file  name,  specify  one  (to  include  the  device  that 
you  wish  to  write  to)  and  hit  <enter>. 

e.  On  an  accompanying  paper  version  of  the  test,  write  down  the  file  n2une 
beside  the  item  for  future  reference. 

4.  Repeat  steps  2  and  3  for  all  of  the  items  in  the  test.  If  you  encounter  a 
matching  or  similiar  question,  treat  it  as  a  single  item. 
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Appendix  C:  Hodifying  Statistic  Prediction  Coefficients 

The  formula  for  predicting  the  standard  deviation  of  a  test  built  with 
Testbank  includes  an  experimentally  derived  coeff icient{K) .  Furthermore,  our 
initial  experiments  showed  that  different  values  for  K  are  needed  for  graded 
reviews  and  for  final  exams.  Within  the  software,  K  is  set  to  0.1571  for 
graded  reviews  and  0.2208  for  final  exams. 

These  values  may  be  changed  by  creating  a  file  that  contains  new  values.  The 
file  must  reside  in  the  Testbank  course  subdirectory  and  be  named: 
TBPRED.DAT.  (Notice  that  this  mechanism  allows  each  course  to  have  its  own 
new  values . ) 

The  TBPRED.DAT  file  must  be  a  standard  ASCII  text  file  with  the  value  for  K 
associated  with  graded  reviews  on  the  first  line  and  the  value  for  the  final 
exam  version  of  K  on  the  second  line.  Below  are  examples  of  the  commands 
needed  to  create  this  file  for  a  course  n^lmed:  Math  141  on  the  C:  drive: 

C:\>COPY  CON: \TESTBANK\KATH141\TBPRED.DAT  <Enter> 

0.16  <Enter> 

0.23  <CTRL-Z>  <Enter> 

The  K  value  for  graded  reviews  is  now  0.16  and  for  final  exams  is  now  0.23. 

For  more  information  on  the  statistics  prediction  formulas,  see  the  Testbank 
Software  Maintenance  Guide. 
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